When capturing and reproducing 3-dimensional images in the prior art, information from one camera of a stereo pair of cameras was depicted as one color (e.g. orange) or band of colors and information from the other camera of the pair was depicted in a complimentary color or color band. When viewing such images through 3-dimensional viewers, such as red/blue glasses, the reproduced image would not be perceived in color.
The orange elements in the picture are only seen through the blue lens, the red lens xe2x80x9cwashing outxe2x80x9d the orange elements. For the same reason, the green-blue elements are only seen through the red lens. Hence, each eye sees only one of the two colored pictures. But because the different colored elements are horizontally shifted in varying amounts, the viewer""s eyes must turn inward to properly view some elements, and turn outward to properly view others. Those elements for which the eyes turn inward, which is what the viewer does to observe a close object, are naturally perceived as close to the viewer. Elements for which the viewer""s eyes turn outward are correspondingly perceived as distant. Specifically, if the blue lens covers the viewer""s right eye, as is generally conventional, then any blue-green element shifted to the left of its corresponding orange element appears to the viewer as close. The element appears closer the greater the leftward shift. Conversely, as a green-blue element is shifted only slightly leftward, not at all, or even to the right of its corresponding red element, that element will appear increasingly more distant from the viewer.
The above mentioned co-pending applications teach techniques for producing color 3-dimensional images.
When 3-dimensional images are captured, corresponding points of the left image are displaced from the same points in the right image horizontally. A measurement of the amount of displacement is called xe2x80x9cdisparityxe2x80x9d. In the prior art when stereo images are made, the disparity for all subject matter visible in both images is fixed. In digital images, disparity can be measured in terms of the number of pixels an object is displaced in the right image relative to its position in the left image. Fixed focal length lenses are customarily used for the cameras.
In an object with zero disparity, the corresponding pixels for the left and right images are perfectly superimposed and the object appears to be located on the screen. Zero disparity objects are seen most clearly when the eyes are crossed just enough to focus on the plane of the screen. Negative disparity objects appear to come out of screen toward the viewer and are seen most clearly when the eyes are more crossed. Positive disparity objects appear to be more distant than the screen and are seen most clearly when the eyes are less crossed.
The eyes cross or uncross in order to get similar image features on or near the fovea of each eye. The xe2x80x9cfarthestxe2x80x9d object that can be seen in an anaglyph is limited by the observers ability to comfortably uncross the eyes. (The usual limit to distant viewing is set by the condition where the eyes look along parallel axes, but such xe2x80x9cwall-eyedxe2x80x9d condition is rarely comfortable to the observer.)
In an anaglyph, the disparity for all objects is fixed and is measured in terms of pixels of displacement.
When one xe2x80x9czooms-inxe2x80x9d on a computer image to see more detail, the pixels get larger and the center-to-center spacing between pixels becomes larger. Therefore, constant disparity (measured in pixels) image components become physically farther apart on the screen. In order for the human visual system to fuse image components and produce the sensation of true stereo vision the eyes have to uncross more for each step of xe2x80x9czoom-inxe2x80x9d. Eventually, the physical separation between corresponding image components becomes so great that the eyes cannot xe2x80x9cuncrossxe2x80x9d comfortably any more (wall-eyed condition) and stereo depth is lost to the observer.
Some stereo images cover such a great range of depth and will have such widely varying values (even without a xe2x80x9czoom-inxe2x80x9d) that some portions of the image will always be out of range of the observer""s ability to see the stereo effects, regardless of how the anaglyph was formed.
Three dimensional techniques are closely related to the psychology and physiology of an observer""s cognitive processes. Subtle changes in selection of portions of the spectrum presented to each eye can result in significant changes in the observer""s perception. Even when viewing the same 3-dimensional image through the same viewers, different observers may perceive a 3-dimensional image in different ways.
The depth location of the point at which the left and right image points for objects at that distance coincided constitutes a xe2x80x9cneutral planexe2x80x9d and when observing a fixed disparity 3-dimensional image, the neutral plane would be found at the surface of the medium of reproduction (i.e. paper or CRT display). Items that appear closer than the medium surface and those points in the image which appear behind the neutral plane would have different disparity. The loss of depth perception when disparity exceeds a certain value generally means that when zooming-in on part of a stereo image pair that disparity will become so great that depth perception will be lost. This is a serious drawback when, for example, attempting to use medical images captured in stereo for instructional purposes. Typically, one would need to examine parts of an object in detail by going close up. This problem is analogous to having a fixed focal length microscope and being unable to see close up features which do not lie directly in the focal plane.
Also in the prior art, when capturing 3-dimensional images on film, magnetic tape or the like, there is no way to visually monitor the combined impact of the separate images being captured. As a result there is no way of adjusting disparity or automatically tracking an object and adjusting disparity automatically.
In the prior art, there is no way to control an image so as to position it either in front of or behind a neutral plane in a controllable fashion. This limits the ability to create 3-dimensional animations.
Also in the prior art, there was no way to adjust the views of 3-dimensional images captured on a static medium, such as CD/ROM.
In the prior art, when viewing stereo images, particularly for extended periods of time, viewers experience a certain amount of discomfort, such as eye strain and headaches. It would be desirable to minimize or eliminate such discomfort.
In the process of zooming in upon an object using stereo cameras, to avoid a loss of stereo effect as disparity exceeds the limits of the human mind to fuse the two images together into a 3-dimensional view, in the prior art cameras were xe2x80x9ctoed inxe2x80x9d toward the object being zoomed upon. The inventors have recognized that this produces certain undesirable results which should be avoided.
When using cameras fixed as to separation and orientation, for scenes that are too far away there may be too little and for scenes too close there may be too much disparity leading to a loss of stereo effect. This, too, is undesirable.
The prior art lacked the ability to zoom-in on portions of a scene when capturing the scene from one location. In order to zoom-in on a scene in the prior art, a stereo camera pair with fixed focal length had to be physically relocated closer to the object being captured.
One advantage of the invention is that it allows for controlling disparity when capturing or reproducing an image.
Another advantage of the invention is that it permits a user to control the disparity by which left and right images are separated.
Another advantage of the invention is the simultaneous adjustment of focal length in stereo camera pairs.
Another advantage of the invention is the ability to adjust camera separation or camera toe-in.
Another advantage of the invention is that it permits zooming-in on portions of a stereo image without losing depth perception.
Another advantage of the invention is the ability to control the location of the neutral plane in 3-dimensional views, thus enabling objects to be controllably placed in front of the neutral plane (popping out of the screen) or behind the neutral plane (in background).
Another advantage of the invention is the ability to create a computer animation using disparity control to produce very realistic animations which move in front of and behind the neutral plane.
Another advantage of the invention is a reduction in the amount of personal discomfort experienced when viewing stereo images. The inventors have discovered that the amount of vertical shift between corresponding points on left and right images can cause discomfort such as eye strain and headaches as one eye tries to move upward vis-a-vis the other to fuse the corresponding points into a stereo image.
Another advantage of the invention is that cameras can be maintained in a parallel orientation without undesirable vertical shift caused by xe2x80x9ctoe inxe2x80x9d of two cameras in an attempt to compensate for exceeding the disparity limits tolerated by the human brain for fusing a stereo image.
Another advantage of the invention has to do with preventing the loss of stereo effect when distances are so far that there is too little disparity for the stereo effect or are too close so that there is too much disparity for the stereo effect.
These and other objects and advantages of the invention are achieved by providing methods and apparatus for viewing three dimensional images which shift one image view with respect to an other image view to control the amount of disparity between corresponding points of the two views and displays the image views so as to form a three dimensional image. The shifting of one image view with respect to another is accomplished by cropping two image planes at different ends by the amount of a desired disparity shift and then combining the cropped image planes to produce a three dimensional display. The shifting can also be accomplished by limiting the read out of certain addresses of each line of video memory image information using a shift register to receiving a line of image data and selecting which cell of the shift register is used for shifting the contents of the shift register to an output.
The invention is also directed to apparatus for capturing and storing three dimensional images using a left camera and a right camera, each with a zoom lens. The zoom lenses are controlled so that each camera zooms substantially identical amounts when zooming is used.
The invention also relates to apparatus for mechanically controlling disparity of images captured by two different cameras, and storing those images. One camera is movably mounted for controlled movement with respect to the other, such as toe-in or horizontal offset.
The invention is further directed to an apparatus for zooming on a scene using a three dimensional camera arrangement with each camera having a zoom lens. The zoom lenses are controlled with servomechanism so that each zoom lens zooms the same amount. A number of coding indications may be used to control the amount of zoom.
The inventions is further directed to apparatus for producing three dimensional images captured using a left and a right video camera connected to respective left and right video recorders. Images from the left and right video recorders are synchronized with each other. The output of the left video recorder is filtered to eliminate blue and green information and the output of the right video recorder is filtered to eliminate the red information. The two outputs are combined to produce a three dimensional image.
The inventions is also directed to reproducing a three dimensional image from first and second digital images stored on a storage medium such as CD/ROM. The first and second digital images are both decomposed into red, green and blue color planes. One color plane of the first digital image is shifted with respect to other color planes of said second digital image and the shifted color plane of the first digital image is combined with the other color planes to produce a three dimensional image.
The invention also contemplates a method of live monitoring of three dimensional images being captured by first and second cameras to a storage medium separating the output of each camera into color planes, combining one color plane of said first camera with one or more different color planes from said second camera; and by displaying the combined color planes.
The invention also contemplates a method of automatically adjusting an image parameter such as disparity during creation of a reproduction of a live scene by placing a small, highly reflective material on a target to be tracked, illuminating said highly reflective material, using the bright spot created by reflection from said highly reflective material for calculating target position and by adjusting said image parameter based on the calculated target position. The image parameter can also be focus or zoom.
The invention is also directed to a method of moving the apparent position of an object represented as left and right images which together constitute a three dimensional image viewed by a viewer to make the object appear to move toward the viewer or recede away from the viewer by shifting the position of the left and right images to change the disparity between the left and right images to thereby cause the perceived relative positions of the object to move
The invention also permits creating a three dimensional computer generated animation of an object by representing said object as a three dimensional wire frame, rendering a surface on said wireframe, creating two color perspective views of said rendered wireframe, separating each of said two color perspective views of said rendered wireframe into 3 color planes, combining one color plane from one of said views with two other color planes from the other view, storing the combined color planes as a three dimensional image, moving said object by modifying said wire frame; and repeating the steps for as many iterations as desired and then displaying sequentially each three dimensional images stored as a three dimensional animation.
In one aspect, the invention is directed to apparatus for capturing three dimensional stereo images of a scene using a left camera and a right camera with parallel optical axes and with a mechanism for maintaining spacing between the cameras at a fixed fraction of a distance to a targeted object. The fixed fraction is a default value which can be overridden by manual adjustment.
The invention is also directed to apparatus for capturing three dimensional stereo images of a scene using a left camera and a right camera with substantially parallel optical axes, and substantially identical zoom lenses. Disparity is adjusted while adjusting focal length of the zoom lenses to hold a location of the neutral plane substantially constant and/or to avoid loss of stereo effect.
The invention is also directed to a method of maintaining disparity of a stereo pair of images within a range which avoids a loss of stereo effect without camera toe in while avoiding vertical shift between corresponding points of a left view and a right view of said stereo pair by maintaining the optical axes of a left camera and of a right camera substantially parallel, and by adjusting disparity while adjusting distance to a target object to avoid loss of stereo effect.
Still other objects and advantages of the present invention will become readily apparent to those skilled in the art from the following detailed description, wherein only the preferred embodiment of the invention is shown and described, simply by way of illustration of the best mode contemplated of carrying out the invention. As will be realized, the invention is capable of other and different embodiments, and its several details are capable of modifications in various obvious respects all without departing from the invention. Accordingly, the drawings and description are to be regarded as illustrative in nature and not as restrictive.